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Abstract.  We  argue  that  metacognition  is  a  critical  component  in  the  development  of 
intercultural  competence  by  highlighting  the  importance  of  supporting  a  learner’s  self- 
assessment,  self-monitoring,  predictive,  planning,  and  reflection  skills.  We  also  survey 
several  modem  immersive  cultural  learning  environments  and  discuss  the  role  intelligent 
tutoring  and  experience  management  techniques  can  play  to  support  these  metacognitive 
demands.  Techniques  for  adapting  the  behaviors  of  virtual  humans  to  promote  cultural 
learning  are  discussed,  as  well  as  explicit  approaches  to  feedback.  We  conclude  with 
several  suggestions  for  future  research,  including  the  use  of  existing  intercultural 
development  metrics  for  evaluating  learning  in  immersive  environments  and  to  conduct 
more  studies  of  the  use  of  implicit  and  explicit  feedback  to  guide  learning  and  establish 
optimal  conditions  for  acquiring  intercultural  competence. 
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Introduction 

Learning  and  adapting  to  a  new  culture  is  a  significant  challenge.  In  different  cultural 
contexts,  interpersonal  and  communicative  behaviors  that  seem  natural  may  produce 
unexpected  results.  For  example,  simple  habits  such  as  nodding  and  other  forms  of 
backchannel  feedback  can  lead  to  unintended  agreements  that  may,  in  turn,  negatively  affect 
trust,  reputation,  and  so  on.  It  is  certainly  important  for  someone  who  will  be  spending  time 
in  a  new  cultural  context  to  prepare  and  be  prepared  for  what  awaits  them.  This  is  the 
problem  cross-cultural  education  programs  attempt  to  solve. 

A  common  approach  is  to  provide  a  learner  with  a  long  list  of  “do’s  and  don’ts”  specific 
to  the  country  or  culture  they  will  be  experiencing.  While  straightforward  and  easy,  this 
approach  relies  heavily  on  rote  learning  and  produces  little  or  no  deep  understanding  of 
culture.  It  also  ignores  empirical  evidence  that  to  develop  intercultural  competence  in  a 
general  way,  people  need  to  move  through  identifiable  stages  of  development  [3,11,13,20]. 
Rushing  to  the  point  of  behavior  adjustment  with  limited  or  no  understanding  of  the 
underlying  cultural  reasons  can  be  problematic.  Tine  intercultural  competence  requires  (at 
least)  a  heightened  sense  of  self-awareness,  an  ability  to  self-assess,  enhanced  perceptive 
abilities,  and  a  proclivity  to  reflect  on  experience.  In  other  words,  intercultural  development 
requires  metacognitive  maturity.  This  paper  is  about  this  process  and  how  immersive 
learning  environments  and  intelligent  tutoring  can  be  used  to  promote  intercultural  learning. 
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1.  Metacognition  in  learning 


Metacognition  involves  active  control  over  cognitive  processes  during  problem  solving.  For 
example,  when  one  is  solving  an  algebra  equation,  cognition  refers  to  the  activities  necessary 
to  solve  it,  such  as  identifying  rules  to  apply,  applying  them,  finding  a  solution,  and  so  on. 
Metacognition  refers  to  a  higher  order  of  thinking  that  operates  on  these  cognitive  activities, 
such  as  planning,  analyzing,  assessing,  monitoring,  and  reflecting  on  problem  solving 
decisions  and  performance.  Metacognition  also  enables  more  effective  learning.  A  learner 
who  is  able  to  accurately  gauge  his  or  her  own  understanding  is  better  equipped  monitor  his 
or  her  own  progress.  This  typically  involves  self-questioning  and  is  part  of  the  larger  notion 
of  metacognitive  regulation  [6]. 

Metacognitive  skills  can  be  taught.  Numerous  classroom  studies  have  shown  that 
explicitly  teaching  metacognitive  strategies  in  the  context  of  a  specific  domain  (e.g.,  physics) 
can  improve  learning  outcomes  [5,  p.  19].  Strategies  taught  in  these  studies  integrate 
metacognitive  activities  with  cognitive  and  seek  to  make  the  steps  of  analyzing,  planning, 
assessing,  and  reflection  habitual  in  the  learner.  Studies  have  also  shown  that  learning  is 
more  effective  when  learners  explain  worked  out  solutions  to  themselves  [7].  This 
phenomenon,  which  better  learners  do  spontaneously,  is  known  as  the  self-explanation  effect. 
It  can  also  be  taught  [8].  More  recently,  computer  tutors  focusing  on  teaching  metacognitive 
skills  have  shown  positive  effects  on  learning  behaviors  (e.g.,  [1]). 


2.  Developing  intercultural  competence 

2. 1.  The  Peace  Corps  model 

Peace  Corps  volunteers  are  told  to  expect  four  levels  of  cultural  awareness  when  they  begin  a 
new  assignment  in  a  foreign  country  [10,  p.  199]: 

1.  Unconscious  incompetence:  minimal  awareness  of  cultural  difference  or  mistakes 
made;  a  state  of  “blissful  ignorance”. 

2.  Conscious  incompetence:  basic  realization  of  cultural  difference;  minimal 
understanding  of  underlying  reasons  or  their  significance. 

3.  Conscious  competence:  increased  understanding  of  differences;  deliberate 
behavioral  adjustments  are  made  to  reduce  cultural  errors  and  misunderstandings. 

4.  Unconscious  competence:  culturally  appropriate  behavior  is  more  or  less 
automatic;  one’s  “instincts  have  been  reconditioned." 

Peace  Corps  educators  have  therefore  recognized  that  a  certain  level  of  metacognitive 
maturity  is  needed  to  become  interculturally  competent.  For  example,  to  move  from 
unconscious  to  conscious  incompetence,  one  must  realize  that  what  seems  “normal”  may  be 
considered  strange  by  people  from  another  culture.  It  must  be  recognized  that  any  peculiar 
observed  behavior  is  likely  in  reaction  to  the  learner.  For  people  with  underdeveloped 
metacognitive  abilities,  there  is  a  real  risk  these  connections  will  fail  to  be  made,  leading 
them  to  conclude  the  strangeness  they  perceive  is  inevitable  and  beyond  their  understanding. 

2.2.  The  Developmental  Model  of  Intercultural  Sensitivity 

The  Peace  Corps  model  is  useful  as  a  teaching  tool,  but  was  not  designed  as  an  explanatory 
scientific  model.  The  Development  Model  of  Intercultural  Sensitivity  (DMIS),  created  by 
Bennett  [3],  is  such  a  model.  It  is  intended  to  explain  how  people  construe  cultural  difference 
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Figure  1.  The  Developmental  Mode  of  Intercultural  Sensitivity  [3], 

and  how  this  ability  becomes  more  flexible  with  time.  By  construe,  Bennett  is  referring  to 
Kelly’s  [16]  constructivist  view  that  experience  is  a  function  of  how  one  assigns  meaning  to 
events  that  occur  in  their  lives.  It  is  not  simply  a  matter  of  being  present  during  some  event 
or  set  of  events,  but  rather  how  those  events  are  interpreted,  encoded  into  memory,  and  later 
remembered.  An  underlying  assumption  of  the  DMIS  is  that  as  one’s  ability  to  construe 
cultural  differences  evolves,  intercultural  competence  also  increases.  According  to  Bennett, 
“it  is  the  construction  of  reality  as  increasingly  capable  of  accommodating  cultural  difference 
that  constitutes  development”  [3,  p.  24]. 

What  constitutes  a  cultural  difference  for  someone?  It  depends  on  that  person’s  cultural 
worldview,  which  is  defined  as  the  set  of  distinctions  the  person  draws  from  to  construe 
events  in  the  world.  A  monocultural  person  -  one  who  has  primarily  experienced  a  single 
culture  in  his  or  her  life  -  will  be  unable  to  construe  perceived  differences  from  outside  that 
cultural  worldview.  On  the  other  hand,  a  person  with  a  broader  understanding  is  generally 
able  to  understand,  even  assume,  other  cultural  worldviews.  Hammer  and  Bennett 
summarize:  “The  crux  of  the  development  of  intercultural  sensitivity  is  attaining  the  ability 
to  construe  (and  thus  to  experience)  cultural  difference  in  more  complex  ways”  [11,  p.423]. 
The  DMIS  is  a  posits  two  broad  worldview  orientations:  ethnocentrism  and  ethnorelativism. 
Each  consists  of  three  stages  and  are  described  below.  The  model  is  summarized  in  figure  1 . 

Bennett  defines  ethnocentrism  as  an  assumption  that  “the  worldview  of  one’s  own 
culture  is  central  to  all  reality”  [3,  p.30].  Further,  an  ethnocentric  person  will  implicitly 
assume  that  all  others  share  this  same  worldview.  The  first  ethnocentric  stage  is  denial  of 
difference  in  which  the  learner  ignores  or  neglects  differences.  The  next  stage  is  defense 
against  difference  which  includes  recognition  of  cultural  difference,  but  with  negative 
evaluation.  This  stage  is  characterized  by  an  “us  vs.  them”  mindset  and  overt,  negative 
stereotyping.  The  last  ethnocentric  stage  is  minimization  of  difference  and  includes  the  first 
signs  of  considering  another  cultural  worldview.  In  this  stage,  the  learner  emphasizes 
similarities  between  cultures  and  recognizes  only  superficial  cultural  differences.  Comments 
such  as  “we  are  all  the  same”  are  common  at  this  stage.  Guidance  is  especially  important 
because  some  learners  believe  minimization  is  the  ultimate  stage  of  growth.  When  reality 
sets  in  that  cultural  differences  are  truly  significant,  there  is  a  risk  of  withdrawal  [3,  p.  44]. 

The  remaining  three  stages  represent  a  shift  to  the  ethnorelative  orientation  and  are 
characterized  by  a  basic  understanding  that  one’s  culture  is  but  one  out  of  many  valid 
worldviews.  The  first  ethnorelative  stage  is  acceptance  of  difference  in  which  the  learner 
recognizes  and  appreciates  cultural  differences.  Cultural  difference  evokes  positive  feelings 
in  the  learner  for  the  first  time.  The  next  stage,  adaptation  to  difference,  is  akin  to  the  Peace 
Corps  “conscious  competence”  stage  in  that  the  learner  makes  an  asserted  effort  to  take  the 
perspective  of  others.  Because  of  this  “frame  shifting”  ability,  the  learner  can  more  easily 
interact  with  people  from  other  cultures.  The  final  stage  is  integration  of  difference',  the 
learner  has  internalized  multiple  cultural  worldviews  and  can  easily  assume  different 
perspectives.  Integration  is  an  advanced  stage  often  requiring  years  of  experience  to  achieve. 


2.3.  Metacognition  and  theDMIS 


Metacognitive  skills  are  critical  for  advancement  through  the  DMIS  stages.  Given  that  the 
model  is  based  on  how  one  construes  cultural  differences,  it  follows  that  a  learner  must 
become  aware  of  the  construal  process  [16],  how  it  works  and  how  to  assess  their  own 
constiual  abilities.  The  following  metacognitive  skills  (at  least)  are  related  to  the  DMIS: 

•  Enhanced  perceptive  abilities  are  needed  to  consciously  recognize  cultural 
differences  without  reacting  to  them  immediately. 

•  Self-assessment  failures  on  one’s  own  interactions  can  hinder  progress  through  the 
DMIS  stages.  This  feeds  into  self-monitoring  and  tracking  through  the  stages. 

•  Cultural  self-awareness,  defined  as  an  understanding  of  one’s  own  culture,  is  an 
important  aspect  of  movement  from  ethnocentrism  to  ethnorelativism  [2], 

•  Self-regidated  learning  is  part  of  intercultural  developmental:  “Generally,  people  in 
the  later  phase  of  adaptation  know  how  to  orchestrate  their  own  learning"  [3,  p.  59]. 

•  Planning  and  goal-setting  can  support  progression  through  DMIS  stages,  such  as 
seeking  to  reach  a  specific  stage  or  to  understand  specific  cultural  differences. 

Bennett  addresses  metacognition  in  his  discussion  of  the  final  stage,  integration  of  difference. 
He  explains  how  one’s  identity  is  not  lost  in  an  ethnorelative  state: 

Rather,  the  integrated  person  understands  that  his  or  her  identity  emerges  from  the  act  of  defining 
identity  itself.  This  self-reflective  loop  shows  identity  to  be  one  act  of  constructing  reality,  similar  to 
others  that  yield  concepts  and  cultures.  By  being  conscious  of  this  dynamic  process,  people  can 
function  in  relationship  to  cultures  while  staying  outside  the  constraints  of  any  particular  one.  (p.60) 

The  risk  is  that  some  learners  may  feel  like  their  culture  and  individuality  is  lost  once  they 
reach  this  advanced  stage.  Bennett’s  point  is  that  the  learner  must  also  accept  this  redefined 
and  more  advanced  understanding  of  self  -  one  that  relies  on  metacognitive  maturity.  Of 
course,  reaching  this  is  well  out  of  the  scope  of  any  educational  approach;  but,  it  does  make  a 
strong  case  for  nurturing  an  intercultural  learner’s  metacognitive  skills.  The  Peace  Corps’ 
approach  is  consistent  with  it  [10],  as  are  other  training  programs  [14,18].  The  rest  of  this 
paper  explores  how  immersive  learning  environments  may  provide  additional  support. 


3.  Intelligent  techniques  for  guided  cultural  learning 

3.1.  Immersive  cultural  learning  environments  and  virtual  humans 

Technologies  such  as  virtual  reality  and  photoreal  3D  graphics  are  particularly  important 
when  considering  cross-cultural  training.  High  fidelity  simulations  make  it  possible  to  create 
realistic  portrayals  of  the  products  of  different  cultures,  such  as  architecture,  dress,  sounds, 
art,  and  even  smells.  This  can  promote  the  learner’s  sense  of  immersion  and  provide  a 
foundation  for  identifying  objective  cultural  differences.  Two  such  environments  are  shown 
in  figure  2.  The  first  is  the  Tactical  Iraqi  Language  and  Culture  Training  System  (TLCTS) 
developed  by  Tactical  Language  Training,  Inc.  [15].  TLCTS  teaches  Arabic  language  and 
cultural  skills.  In  the  mission  environment  (shown  in  the  screenshot),  a  learner  is  free  to 
explore  an  Iraqi  village,  hear  the  sounds,  speak  to  locals,  and  make  gestures.  The  clothing, 
buildings,  and  surroundings  are  realistic  and  thus  can  give  a  learner  a  sense  of  what  it  might 
be  like  to  walk  around  an  Iraqi  village.  In  this  way,  the  system  is  already  in  a  position  to  aid 
in  the  learner’s  identification  of  cultural  differences. 


Figure  2.  Examples  of  immersive  cultural  learning  environments:  (left)  Tactical  Iraqi  Language  and  Culture 
Training  System  [15]  and  (right)  the  Adaptive  Thinking  and  Leadership  simulation  [21].  Used  with  permission. 


The  screenshot  on  the  right  in  figure  2  is  from  the  Adaptive  Thinking  and  Leadership 
(ATL)  simulation  game  [21].  ATL  is  a  team-training  system  that  uses  human  role  players  for 
both  sides  in  intercultural  scenarios.  In-game  assessment  is  performed  by  peers  and 
instructors  who  observe  play  and  after-action  review  (AAR)  facilities  are  available  to  convey 
the  outcomes  to  trainees.  Learners  are  often  assigned  to  role  play  as  people  from  different 
cultures,  with  appropriate  backstories  and  goals.  Role-playing  is  a  well-developed  technique 
in  the  crosscultural  training  literature  [18]  and  consistent  with  the  DMIS  with  respect  to  the 
goal  of  understanding  different  cultural  worldviews. 

In  multi-player  environments,  like  the  ATL  system,  inhabitants  are  human  roleplayers. 
This  can  be  costly  and  sometimes  challenging  to  control  from  an  educator’s  point  of  view. 
Research  in  virtual  humans  provides  an  alternative  or  supplement  to  cultural  team-training  in 
immersive  learning  environments.  Virtual  humans  combine  artificial  intelligence  (AI) 
research  in  cultural  and  emotional  modeling,  speech  processing,  dialogue  management, 
natural  language  understanding,  and  gesturing,  among  others,  to  enable  natural  feeling 
communication  and  interaction  with  computer-controlled  characters  that  listen  and  respond 
to  the  user.  Virtual  humans  are  driven  by  rich  models  of  tasks,  emotion,  body  language,  and 
communication  [22].  The  underlying  representations  readily  support  explanation,  which  can 
be  useful  for  learning  [9].  In  the  case  of  intercultural  education,  it  is  therefore  important  to 
endow  virtual  humans  with  models  of  culture  and  the  ability  to  explain  their  actions  and 
reactions  in  terms  of  their  cultural  worldview.  It  is  also  important  that  their  behavior  be 
controllable  in  order  to  establish  conditions  that  best  promote  learning. 

3.2.  Experience  manipulation  and  implicit  feedback 

Generally  speaking,  computer  simulations  simulate  real  world  phenomena  as  accurately  as 
possible.  There  are  circumstances  when  it  is  appropriate  to  consider  goals  other  than  fidelity 
when  deciding  how  a  simulation  should  behave  and  what  events  should  occur.  For  instance, 
to  enhance  entertainment  value,  a  popular  basketball  video  game  includes  special  modes  that 
allow  players  to  jump  well  over  ten  feet  high.  In  this  case,  the  goal  of  entertaining  the  human 
player  trumps  the  goal  of  simulating  basketball  completely  realistically.  In  the  case  of 
learning,  the  same  idea  applies:  if  a  certain  event  or  situation  will  promote  learning,  then  the 
simulation  should  seek  to  make  that  event  happen.  We  refer  to  this  general  technique  as 
experience  manipulation  and  now  discuss  several  ways  it  might  be  used  to  promote 
metacognitive  growth  and  cultural  learning. 


Figure  3.  Culturally  influenced  expressions  of  anger,  skepticism,  and  appreciation  by  virtual  humans  [9,22]. 


When  a  cultural  error  is  committed,  or  when  appropriate  actions  are  taken,  learners  need 
support  in  (at  least): 

•  recognizing  that  an  error  was  committed  (or  that  a  good  action  was  taken) 

•  finding  a  causal  link  between  the  action  taken  and  the  observed  reaction 

•  understanding  the  reason) s)  and  culpable  underlying  cultural  differences 

•  learning  how  to  avoid  the  same  mistake  in  the  future  (or  sustain  good  actions) 

It  is  important  to  go  beyond  simply  concluding  to  avoid  the  same  behavior  in  the  future  since 
this  will  contribute  little  in  the  learner’s  progression  through  the  DMIS  stages.  Also,  the 
stage  a  person  is  in  impacts  how  cultural  differences  are  interpreted.  Someone  in  the  denial 
phase  may  not  even  be  willing  to  accept  the  fact  that  a  cultural  error  even  occurred,  for 
example.  Someone  in  the  other  two  ethnocentric  stages  (defense  and  minimization)  may  be 
aware  an  error  occurred,  but  unwilling  to  take  blame  or  perhaps  place  the  onus  on  the  virtual 
human  to  be  the  one  who  should  adapt.  Based  this  understanding  of  cultural  growth,  the 
reaction  of  a  virtual  human  to  a  cultural  error  should  be  appropriate  for  that  learner. 

Feedback  from  the  simulation  itself,  such  as  the  oral  and  gestural  reactions  of  virtual 
humans,  is  called  implicit  feedback.  To  support  recognition  of  cultural  errors,  there  are 
several  strategies  that  can  be  used  adjust  implicit  feedback  to  promote  learning.  One  of  the 
simplest  is  to  accentuate  verbal  responses  of  characters  to  draw  more  attention  to  anger  or 
negative  feelings,  in  the  case  of  an  error.  Similarly,  implicit  positive  feedback  can  be 
achieved  by  accentuating  positive  and  laudatory  responses  to  correct  user  actions.  In  some 
cases,  it  may  even  be  appropriate  for  the  virtual  human  to  deliver  an  impassioned  mini¬ 
lecture  regarding  the  cultural  issues  in  question.  The  choice  of  words  by  the  virtual  human 
can  be  designed  to  refer  directly  to  actions  taken  by  learner  to  support  the  pedagogical  goal 
of  linking  cause  and  effect  in  the  learner’s  mind.  In  addition,  the  virtual  human  might  also 
drop  hints  regarding  the  underlying  cultural  differences.  Body  language  and  gestures  can 
have  a  dramatic  effect  on  the  communicative  power  of  utterances.  Figure  3  shows  several 
virtual  humans  in  different  emotional  states  and  displaying  a  variety  of  gestures.  The  timing 
and  emphasis  of  these  gestures  can  be  adjusted  to  meet  pedagogical  goals  in  a  way  similar  to 
the  utterance  content.  Aside  from  body  language,  other  features  that  might  be  adjusted  in 
virtual  humans  are  facial  expressions,  speech  rate,  intonation,  and  tone,  emotional  state,  and 
personality  traits. 


3.3.  Experience  management  and  interactive  narrative 


The  techniques  described  in  the  previous  section  all  focused  on  emphasizing  specific  details 
of  interacting  with  virtual  humans  to  support  the  learner  in  recognizing  cultural  difference 
and  improving  their  ability  to  self-assess  in  an  interpersonal  context.  Of  course,  explanations 
for  why  certain  behaviors  are  culturally  offensive  can  be  very  complex.  They  may  involve 
fundamental  differences  between  worldviews,  varying  ethical  standards,  social  structure, 
historical  and  geographical  factors,  and  so  on.  A  deep  understanding  of  culture  includes 
these  advanced  notions  and  may  enable  a  learner  to  go  beyond  rote  learning  by  providing  the 
knowledge  needed  to  reconstruct  appropriate  surface  behaviors  later.  Cultural  simulations 
should  provide  diverse  cultural  experiences  that  go  beyond  one-on-one  interactions  and 
carefully  manage  how  events  are  presented  and  experienced  by  the  learner. 

One  such  technology  focusing  on  experience  management  is  automated  story’  directing 
(ASD)  [23].  The  goal  is  similar  to  that  of  a  traditional  tutoring  system:  allow  users  to  feel  as 
much  freedom  as  possible,  but  keep  them  on  certain  paths  that  consist  of  certain  experiences. 
The  “path”  in  an  interactive  storytelling  system  is  a  storyline  consisting  of  plots,  arcs,  events, 
and  other  narrative  elements.  Users  may  “break”  a  storyline  by  taking  actions  in  the  virtual 
world,  and  so  ASD  systems  use  a  variety  of  techniques,  like  reactive  planning,  to  repair 
storylines  and  re -plan  when  new  events  are  deemed  desirable.  Often,  the  aim  is  to  maximize 
engagement.  For  cultural  training,  the  additional  aim  is  to  create  situations  and  conditions 
that  promote  practice  and  learning. 

Metacognition  comes  into  play  when  we  consider  the  learner’s  role  in  the  narrative.  She 
or  he  must  be  aware  that  the  actions  being  taken  are  being  observed  by  the  AI  agents  in  the 
simulation  and  that  choices  being  made  have  observable  outcomes.  Just  as  minute  details  of 
interaction  can  be  manipulated  to  highlight  differences,  so  can  story  elements.  For  example, 
if  a  learner  makes  a  gender-related  error  early  in  a  game,  the  ASD  may  decide  to  propagate 
this  knowledge  through  the  social  network  to  force  the  learner  to  enter  future  encounters  with 
this  baggage.  Flere  the  goal  is  not  only  to  teach  gender  specific  cultural  differences,  but  also 
to  encourage  consideration  of  unintended  cultural  consequences  of  earlier  actions.  This 
requires  the  metacognitive  abilities  to  self-assess  over  an  extended  period,  reaching  back 
further  than  just  the  most  recent  action,  and  the  ability  to  predict  (another  metacognitive 
skill).  After  dealing  with  negative  consequences  of  actions,  it  is  hoped  that  a  learner  will 
become  more  likely  to  consider  possible  unintended  outcomes  of  actions  before  taking  them. 

3.4.  Guidance  and  feedback 

Inherent  risks  exist  in  unguided  environments,  such  as  inefficient  learning,  the  formation  of 
misconceptions,  and  development  of  incomplete  or  fragmented  knowledge  [17].  Experience 
manipulation  and  implicit  feedback  can  certainly  mitigate  these  risks  to  a  certain  degree,  but 
to  adequately  address  the  needs  of  novice  and  intermediate  learners,  explicit  feedback  from  a 
human  tutor,  pedagogical  agent,  disembodied  coach,  or  other  form  of  intelligent  tutor  has  the 
potential  to  greatly  enhance  learning  (e.g.,  [4]).  Explicit  guidance  can  come  in  different 
forms  in  an  immersive  learning  environment.  A  pedagogical  agent  who  plays  a  role  in  the 
underlying  simulation  is  a  popular  approach.  The  TLCTS  [15]  and  the  mission  rehearsal 
exercise  described  in  [22]  both  provide  pedagogical  agents  in  the  form  of  a  knowledgeable 
Sergeant  who  maintains  an  understanding  of  the  cognitive  goals  and  can  give  hints  on  how 
to  succeed.  Another  form  is  that  of  a  disembodied  tutor  that  posts  messages  in  a  special  area 
of  a  GUI  or  via  speech.  No  matter  what  the  modality,  explicit  feedback  provides  more  direct 
and  understandable  guidance  than  implicit  -  this  is  especially  important  for  novices  [17]. 


Immersive  learning  environments  can  be  overwhelming  at  times.  With  respect  to 
cultural  learning,  explicit  hinting  and  feedback  can  help  learners  in  several  ways: 

1 .  confirm  a  learner’s  interpretation  of  observed  virtual  humans  behaviors 

2.  explain  the  cultural  differences  in  play  during  specific  interactions 

3.  explain  the  “under  the  hood”  reasoning  of  a  virtual  human 

4.  hint  about  ideal  actions  to  take  or  warn  against  certain  risks 

5.  suggest  the  learner  identify  possible  outcomes  and  desirable  end  states 
Explicit  tutorial  feedback  removes  a  level  of  interpretation  for  the  learner.  Rather  than 
guessing  or  inferring  the  cognitive  and  emotional  states  of  virtual  humans,  a  clear  statement 
by  a  tutor  can  act  as  a  strong  scaffold  for  learning  in  immersive  cultural  environments.  There 
are  certainly  cognitive  aspects  to  the  tactics  listed  above,  but  they  also  address  the 
metacognitive  demands  of  intercultural  development.  Tactics  1-3  encourage  self-assessment 
by  describing  the  impact  of  a  learner’s  actions  on  a  virtual  human.  Because  this  is  feedback 
being  delivered  in  a  real-time  environment,  the  issues  of  distraction  and  cognitive  overload 
need  to  be  considered.  Thus,  it  is  ideal  to  keep  “in-game”  feedback  short  and  precise,  saving 
the  longer  explanations  for  post-practice  reflection.  This  is  the  approach  taken  in  the  ELECT 
Bi-Lateral  negotiation  game  [12]).  Hinting  (tactic  4)  can  be  direct  (and  at  the  cognitive  level), 
but  also  can  be  used  to  encourage  the  learner  to  think  about  pros  and  cons  of  taking  different 
actions  -  this  is  especially  important  in  ill-defined  domains  where  assessment  is  inherently 
challenging  [19].  The  content  of  tactics  1-4  are  precisely  the  things  we  want  the  learner 
speculating  on  before  acting  in  the  environment.  In  other  words,  the  ultimate  aim  is  for  the 
learner  to  self-guide  in  these  precise  ways.  Such  cognitive  activities  by  the  learner  would 
constitute  attempts  at  self-explanation.  Tactic  5,  identifying  potential  end  states,  is  a  purely 
metacognitive  tactic  that  is  geared  towards  supporting  goal  formation  and  identifying  “what 
right  looks  like.”  Encouraging  the  learner  to  “think  before  acting”,  to  engage  in  planning  and 
simulate  hypothetical  actions,  and  “reflect  after  acting”  are  at  the  core  of  metacognitive 
maturity  and  a  fundamental  requirement  for  growth  through  the  DMIS  stages. 

Tutorial  sub-dialogues  are  rarely  possible  in  real-time  environments,  and  so  there  is  time 
only  for  very  brief  periods  of  reflection.  However,  once  a  practice  session  or  exercise  is 
complete,  there  is  time  to  carefully  target  metacognitive  skill  development.  Immersive 
learning  environments  should  therefore  provide  supporting  tools  such  as  video  playback. 
Reflective  tutoring  is  an  appropriate  supplement  to  guide  the  use  of  these  tools  and  to  fill  in 
the  gaps  from  feedback  that  was  delivered  during  practice.  The  reflective  tutoring  system 
built  for  the  virtual  humans  [9,12]  walks  the  student  through  three  questions:  (1)  What 
happened?  (2)  Why  did  these  events  occur  the  way  they  did?  and  (3)  How  can  good 
performance  be  sustained  and  poor  performance  be  improved?  A  promising  approach  here  is 
to  leverage  explanation  facilities  of  virtual  humans  to  uncover  their  thinking  via  explanation 
[9]  and  discover  what  other  actions  may  have  produced  better  outcomes.  An  advanced  tactic 
is  to  restart  the  simulation  to  give  the  learner  a  second  chance  (a  “mulligan”).  Reflection  at 
this  point  may  enhance  self-assessment  skills  and  intercultural  growth. 


4.  Conclusions  and  suggestions  for  future  research 

This  paper  has  argued  that  achieving  intercultural  competence  requires  strong  metacognitive 
abilities.  Although  cross-cultural  training  programs  frequently  adopt  metacognitive 
approaches  to  teaching  intercultural  competence,  the  connection  is  rarely  made  explicit.  The 
Peace  Corps  approach  is  an  example  of  this  that  describes  growth  as  a  continuum  from 
unconscious  incompetence  (not  knowing  anything  and  being  blissfully  unaware  of 


differences)  to  unconscious  competence  (full  awareness  of  differences  and  appropriate 
behavior  is  second  nature).  By  describing  these  stages,  the  learner  put  in  a  position  to  self¬ 
monitor  their  advancement.  This  then  requires  application  of  other  metacognitive  skills,  such 
as  self-assessment,  self-explanation  and  self-regulation,  to  progress  through  the  stages.  The 
Developmental  Model  of  Intercultural  Sensitivity  (DMIS)  is  an  empirically  derived  and 
validated  model  of  intercultural  development  based  on  notion  of  how  cultural  differences  are 
“construed”  by  a  learner  [3,11].  To  develop  intercultural  competence,  it  is  necessary  to 
construe  cultural  difference  in  progressively  more  complex  ways,  such  as  from  different 
cultural  worldview  perspectives.  Growth  here  also  requires  mature  metacognitive  abilities 
and  it  may  be  possible  to  promote  these  skills  in  modem  immersive  learning  environments 
through  a  combination  of  experience  manipulation  and  explicit  guidance.  The  DMIS 
suggests  cultural  difference  as  the  pivot  point  for  intercultural  growth,  and  so  careful 
direction  of  virtual  humans  and  delivery  of  feedback  that  targets  self-assessment,  predictive, 
and  reflection  skills  has  the  potential  to  speed  the  growth  to  intercultural  competence. 

Early  evaluations  of  TLCTS  [15]  and  the  ELECT  BiLAT  [12]  serious  games  have 
shown  some  promising  results  with  respect  to  learning  aspects  of  specific  cultures.  Most  of 
the  computer  simulations  built  for  cultural  education  have  not  undergone  rigorous 
experimental  evaluations  for  learning  or  for  intercultural  development.  Hammer  and 
Bennett’s  Intercultural  Development  Inventory  (IDI)  [11]  has  been  used  to  validate  the 
DMIS  and  may  provide  a  suitable  metric  for  determining  the  value-added,  if  any,  that  comes 
from  augmenting  cultural  training  programs  [18]  with  immersive  learning  environments  - 
especially  if  the  IDI  can  be  administered  repeatedly  and  rates  of  change  can  be  tracked. 
Other  more  general  questions  about  feedback  are  suggested  for  further  study.  For  example, 

•  Does  implicit  pedagogical  feedback  break  immersion? 

•  If  so,  what  is  the  cost  (if  any)  to  breaking  immersion  with  respect  to  learning? 

•  What  is  the  interplay  between  implicit  and  explicit  feedback? 

•  What  situations  merit  the  use  of  explicit  feedback? 

The  use  of  implicit  feedback  and  experience  manipulation  is  perhaps  one  of  the  most 
important  open  questions  to  address.  The  instructor  interface  to  the  ATL  serious  game  [21] 
provides  the  ability  to  throw  “curve  balls”  to  teams  during  their  missions,  such  as  helicopter 
fly-overs.  These  are  intended  to  support  the  development  of  adaptive  thinking  skills  under 
stress.  This  is  related  to  research  in  the  ITS  community  on  intelligent  problem  selection  and 
finding  the  appropriate  level  of  challenge  for  a  learner.  These  connections  need  further 
exploration,  as  does  the  reasoning  behind  expert  instructors’  decisions  to  throw  curve  balls: 
What  are  the  triggers?  How  do  instructors  decide  which  curve  ball  to  throw?  The  answers 
may  not  always  involve  metacognitive  skills,  but  as  this  paper  has  attempted  to  argue, 
manipulations  of  this  sort  in  an  intercultural  context  may  be  ideal  to  highlight  cultural 
differences  to  give  learners  practice  in  dealing  with  them.  This  may  enhance  the 
development  of  intercultural  competence,  but  empirical  research  is  certainly  needed  to 
confirm  this  hypothesis. 


Acknowledgements 

I  thank  Dr.  Jonathan  Kaplan  and  Dr.  Jim  Belanich  of  the  U.S.  Army  Research  Institute  for 
many  interesting  conversations  on  experience  manipulation,  implicit  feedback,  and  difficulty 
that  seeded  the  ideas  presented  in  this  paper.  The  project  described  here  has  been  sponsored 
by  the  U.S.  Army  Research,  Development,  and  Engineering  Command  (RDECOM). 


Statements  and  opinions  expressed  do  not  necessarily  reflect  the  position  or  the  policy  of  the 
United  States  Government,  and  no  official  endorsement  should  be  inferred. 


References 

[1]  V.  Aleven,  B.  McLaren,  I.  Roll,  &  K.  Koedinger.  Toward  meta-cognitive  tutoring:  A  model  of  help-seeking 
with  a  cognitive  tutor.  Int.  Journal  of  Artificial  Intelligence  in  Education ,  16: 101-130, 2006. 

[2]  J.  M.  Bennett.  Intercultural  communication  training  in  cultural  self-awareness  for  study  abroad.  PhD  thesis, 
University  of  Minnesota,  October  1985. 

[3]  M.  J.  Bennett.  Towards  ethnorelativism:  A  developmental  model  of  intercultural  sensitivity.  In  R.  M.  Paige,  Ed., 
Education  for  the  Intercultural  Experience ,  21-71.  Intercultural  Press,  Yarmouth,  ME,  1993. 

[4]  B.  S.  Bloom.  The  2  sigma  problem:  The  search  for  methods  of  group  instruction  as  effective  as  one-to-one 
tutoring .  Educational  Researcher,  13:4—16, 1984. 

[5]  J.  D.  Bransford,  A.  L.  Brown,  and  R.  R.  Cocking,  editors.  How  People  Learn:  Brain,  Mind,  Experience,  and 
School.  National  Academy  Press,  Washington,  DC,  2000. 

[6]  A.  L.  Brown.  Metacognition,  executive  control,  self-regulation,  and  other  more  mysterious  mechanisms.  In  F.  E. 
Weinert  and  R.  H.  Kluwe,  editors,  Metacognition,  motivation,  and  understanding ,  pages  65-116.  Lawrence 
Erlbaum  Associates,  Hillsdale,  New  Jersey,  1987. 

[7]  M.  T.  H.  Chi,  N.  de  Leeuw,  M.  H.  Chiu,  and  C.  LaVancher.  Eliciting  self-explanations  improves  understanding. 
Cognitive  Science,  18(3):43 9-All,  1994. 

[8]  M.  T.  H.  Chi,  S.  Siler,  H.  Jeong,  T.  Yamauchi,  and  R.  G.  Hausmann.  Learning  from  tutoring:  A  student- 
centered  versus  a  tutor-centered  approach.  Cognitive  Science,  25:471-533, 2001. 

[9]  M.  G.  Core,  D.  Traum,  H.  C.  Lane,  W.  Swartout,  J.  Gratch,  M.  vanLent  &  S.  Marsella.  Teaching  negotiation 
skills  through  practice  and  reflection  with  virtual  humans.  SIMULATION,  82(1 1):685-701, 2006. 

[10]  Peace  Corps.  Culture  matters:  The  peace  corps  cross-cultural  workbook,  1999. 

[11]  M.  R.  Hammer,  M.  J.  Bennett,  and  R.  Wiseman.  Measuring  intercultural  sensitivity:  The  intercultural 
development  inventory.  International  Journal  of  Intercultural  Relations,  27:421^-43, 2003. 

[12]  R.  W.  Hill,  J.  Belanich,  H.  C.  Lane,  M.  G.  Core,  M.  Dixon,  E.  Forbell,  J.  Kim,  and  J.  Hart.  Pedagogically 
Structured  Game-based  Training:  Development  of  the  ELECT  BiLAT  Simulation.  In  Proceedings  of  the  25th 
Army  Science  Conference,  2006. 

[13]  D.  S.  Hoopes.  Intercultural  communication  concepts  and  the  psychology  of  intercultural  experience.  In  M.  D. 
Pusch,  editor,  Multicultural  education:  A  cross  cultural  training  approach,  pages  10-38.  Intercultural  Network, 
Inc.,  La  Grange  Park,  IL,  1979. 

[14]  D.  S.  Hoopes  and  M.  D.  Pusch.  Teaching  strategies:  The  methods  and  techniques  of  crosscultural  training.  In  M. 
D.  Pusch,  editor,  Multicultural  education:  A  cross  cultural  training  approach,  pages  104—204.  Intercultural 
Network,  Inc.,  La  Grange  Park,  IL,  1979. 

[15]  W.  L.  Johnson,  H.  Vilhjalmsson,  and  S.  Marsella.  Serious  games  for  language  learning:  How  much  game,  how 
much  AI?  In  C.-K.  Looi,  G.  McCalla,  B.  Bredeweg,  and  J.  Breuker,  editors,  Proceedings  of  12th  International 
Conference  on  Artificial  Intelligence  in  Education,  pages  306-313.  IOS  Press,  2005. 

[16]  G.  A.  Kelly.  Theory  of personality:  The  psychology  of personal  constructs.  Norton,  New  Y  ork,  1 963 . 

[17]  P.  A.  Kirschner,  J.  Sweller,  and  R.  E.  Clark.  Why  minimal  guidance  during  instruction  does  not  work:  An 
analysis  of  the  failure  of  constructivist,  discovery,  problem-based,  experiential,  and  inquiry-based  teaching. 
Educational  Psychologist,  41(2):75-86, 2006. 

[18]  M.  Mendenhall,  G.  Stahl,  I.  Ehnert,  G.  Oddou,  J.  Osland,  and  T.  Kuhlmann.  Evaluation  studies  of  cross-cultural 
training  programs:  A  review  of  the  literature  from  1988  to  2000.  In  D.  Landis,  J.  M.  Bennett,  and  M.  J.  Bennett, 
eds.,  Handbook  of  intercultural  training,  pp.  129-144.  Sage,  Inc.,  Thousand  Oaks,  CA,  2004. 

[19]  A.  Ogan,  V.  Aleven,  and  C.  Jones.  Culture  in  the  classroom:  Challenges  for  assessment  in  ill-defined  domains. 
In  Proceedings  of  the  Workshop  on  Intelligent  Tutoring  Systems  for  Ill-Defined  Domains  at  Intelligent  Tutoring 
Systems.  Jhongli,  Taiwan,  2006. 

[20]  R.  M.  Paige,  M.  Jacobs-Cassuto,  Y.  A.  Yershova,  and  J.  DeJaeghere.  Assessing  intercultural  sensitivity:  An 
empirical  analysis  of  the  Hammer  and  Bennett  Intercultural  Development  Inventory.  International  Journal  of 
Intercultural  Relations,  27:467-486, 2003. 

[21]  E.  M.  Rayboum.  Applying  simulation  experience  design  methods  to  creating  serious  game-based  adaptive 
training  systems.  Interacting  with  Computers,  19:206-214, 2007. 

[22]  W.  Swartout,  J.  Gratch,  R.  Hill,  E.  Hovy,  S.  Marsella,  and  J.  Rickel.  Toward  virtual  humans.  AI  Magazine, 
27(2):96-108,  July  2006. 

[23]  R.  M.  Young.  Towards  an  Architecture  for  Intelligent  Control  of  Narrative  in  Interactive  Virtual  Worlds.  In 
Proceedings  of  the  2003  International  Conference  on  Intelligent  User  Interfaces.  Miami,  FL,  2006. 


